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ABSTRACT 



In order to evaluate and utilize library data for the 



management process, a German project, sponsored by the German Research 
Council, uses the Balanced Scorecard as the concept for integrated quality 
management. Performance indicators across the following four perspectives are 
combined to produce a balanced evaluation of the library: (1) users, 

including market penetration, user satisfaction rate, user satisfaction with 
opening times, library visits per member of the population, immediate 
availability, percent of the population using electronic library services, 
and percent of remote accesses to electronic library services of all 
accesses; (2) finances, including total costs of the library per active user, 
total costs of the library per library visit, acquisitions expenditure 
compared to the total costs of the library, and percent of staff costs per 
library service/product to total staff costs; (3) internal processes, 
including acquired media per staff year, average media processing time, 
number of stages involved in providing a product/service, and percent of all 
staff costs spent on electronic services; and (4) potentials (innovation) , 
including library budget as a percent of the institution’s budget, percent of 
the library's budget from project funding or income generation, number of 
training lessons per staff member, and number of short-time illnesses per 
staff member. (MES) 
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Libraries are collecting scores of data about their collection, services, use, and costs and about the 
quality of their services and products. In order to evaluate and utilize these data for the management 
process, a systematic approach is needed. 

A German project, sponsored by the German Research Council, uses the Balanced Scorecard as concept 
for an integrated quality management. Performance indicators across four equally significant 
perspectives - users, finances, internal processes, and potentials (innovation) - are combined to produce 
a " balanced " evaluation of the library. 
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The data in the controlling system 

Traditionally, libraries have collected statistical data about their collections, acquisitions, lending, and 
interlending activities. In time, these statistics were enlarged and differentiated, and in many cases now 
comprise several hundreds of separate data, ranging from the number of incunabula or microforms in the 
collection, expenditure on preservation or buildings to the number of issues, claims and reservations or the 
visits to exhibitions and special events. These statistics are in the most part collected nationally, but 
libraries tend to collect other statistics additionally, e. g. for special tasks and activities like legal deposit 
right, special collections, or services for special user groups. 

All those statistical data could be used as steering instruments for library management, but more often 
than not such use is rather accidental than systematical, and many data are collected laboriously without 
ever being evaluated. 
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Far several decades libraries have tried to assess not only the quantity of their resources and activities, but 
also the quality, the "goodness" of a library’s services and products. Performance measures for libraries 
have been developed and tested in national and international projects and standardized in an international 
standard 1 . Though there are lists of recognized and established performance indicators, such indicators 
have not been collected systematically, like the national collection of statistics, but rather in the evaluation 
process of a single library. 

Both statistics and performance indicators were developed for the traditional library, with print 
collections, reading rooms and lending services. The growing importance of electronic services in libraries 
has led to a revision of both statistics and performance indicators. The international standard of library 
statistics has been revised and enlarged to include the data of the "digital library" 2 , and a working group of 
ISO (International Organisation of Standardization) is drafting a Technical Report about performance 
indicators for electronic library services 3 

Another sector of management data has evolved during the last years: Cost data. Libraries have always 
registered data of their income and expenditure. But the general demand for transparency of costs has led 
to questions like: 

What are the costs of each single service or product of a library? (e. g. one issue, one reference 
question answered) 

How do the costs of a service or product split up as to staff costs, administrative costs, equipment etc.? 

More and more libraries are involved in cost analysis projects of their institutions, or are trying to analyse 
their costs in order to present reliable data when applying for funds or allocating resources. Models for 
cost analysis in libraries have been tested and developed 4 and will probably be used widely in the future. 

There is an immense pool of management data available today in libraries: Statistics of resources, 
services, and use, cost data and combined data like performance indicators for the quality of library 
services. The quantity, diversity and complexity of the data stresses the need for an integrated system to 
make their management useful for evaluation, strategy and action. 

The project 

A German project, sponsored by the German Research Council (DFG), has developed an integrated 
quality management system for academic libraries. The project is chaired by the University and Regional 
Library Munster, partners are the Bavarian State Library Munich and the State and University Library 
Bremen. The three libraries are among the largest in Germany, each with special tasks, activities, and 
operating conditions. Thus the project could rely on a broad and differentiated view of management issues 
in academic libraries. The project started in June 1999 and will be finished in autumn 2001. The results 
will be published in a handbook and a software for the data collection and the management process will be 
added. 

The project partners decided to use the Balanced Scorecard 5 as tool for the management system. This 
concept was originally developed for the commercial sector. It "translates" the planning perspective of an 
institution (mission, strategic vision and goals) into a system of performance indicators that covers all 
important perspectives of performance: finances, users, internal processes and improvement activities. 



The system thus integrates: 

financial and non-financial data, 



input and output data, 

the external perspective (funding institutions, users) and the internal perspective (processes, 
staff), 

goals and measures taken, 
causes and results. 



The basic model of the Balanced Scorecard, adapted to the conditions of academic libraries, deviates from 
the original model in placing not the financial, but the user perspective foremost. Libraries do not strive 
for maximum gain, but for best service. 

Figure 1 : The Balanced Scorecard 

The user perspective 

The indicators chosen for the user perspective correspond to the fundamental goals of reaching as large a 
part as possible of the population and of satisfying their informational needs by the services offered. 

• Market penetration = Percent of the population registered as actual users 

For an academic library, its population to be served will probably consist of the members of 
the institution (university, scientific institute etc.) it is set up to serve. 

It would, of course, be still more interesting to asses the members of the population who made 
use of library services during a specified time, including electronic services and remote use. 
But this, for many libraries, is still difficult to measure. 

• User satisfaction rate 

The indicator is assessed by satisfaction surveys on a 5-point-scale. 

• User satisfaction with opening times 

Again, this indicator is assessed by surveying. 

• Library visits per member of the population 

The visits to the library premises (counted by turnstile, electronic counter etc.) are compared 
to the number of members of the population. 

"Virtual visits" (cases of remote use) could be included if libraries can differentiate remote use 
by members of the population from other cases of use. 

• Immediate availability: Percent of immediate loans of total number of loans (including 
reservations and ELL) 

The indicator shows whether the collection covers all topics asked for by users and whether 
there are sufficient copies. 

Two indicators assess the use of electronic services offered by the library and the growing portion of that 
use coming from outside the library building: 

• Percent of the population using the electronic library services 

This indicator is assessed by a survey. Electronic services include the OP AC, the library’s 
website, electronic databases, journals, and other documents, and electronic document 
delivery. 
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• Percent of remote accesses to electronic library services of all accesses 

This indicator assesses to what amount library services are used from outside the library. One 
access (or: log-in, session) is one established connection to an electronic library service. 

The financial perspective 

The indicators for the financial perspective answer to the question whether the library is functioning in a 
cost-effective way. The goals comprise low costs per instance of use or per product and a high proportion 
of the total budget spent on the print and electronic collection. 

• Total costs of the library per active user 

• Total costs of the library per library visit 

• Acquisitions expenditure compared to the total costs of the library 

• Percent of staff costs per library service / product to total staff costs 
A last indicator shows the allocation of resources to the electronic library: 

• Percent of acquisitions expenditure spent on electronic media 
The perspective of processes 

For the perspective of processes, the underlying goals are to organize all processes in a way that in spite 
of budget restrictions allows space for investment into new developments and improvement of service. 
The indicators pick out background activities as examples of process organisation. 

• Acquired media per staff year 

Staff in the processing department is counted in FTE (full time equivalent) 

The processing department comprises all activities from acquisition and cataloguing to 
binding and shelving the new media. 

Electronic media are not included in this indicator. 

Book processing is here seen as a typical example for the effectiveness of background 
processes. 

• Average media processing time 

• Number of stages involved in providing a product / service 

(for every library service / product) 

The indicator tries to assess, whether processes are streamlined and well organized. Typical 
examples are collection building and document delivery services. 

Again, one indicator was chosen to show the allocation of resources to the electronic services: 

• Percent of all staff costs spent on electronic services 

Library staff planning, maintaining, providing, and developing electronic services and 
providing and cataloguing electronic media is calculated in FTE (full time equivalent). The 
definition could also comprise staff in user services, e. g. reference and training services, 
dealing with electronic media, and the staff involved in the contents of the website. 



The perspective of potentials 



The last perspective, named ’’potentials”, describes the capability of the library to cope with the 
challenges of the future, its ability to change and improve. The institution’s engagement for the library is 
indicated by the budget it allocates to the library; staff as the main factor for all development is 
represented by two indicators for teaching and engagement. 

• Library budget as percent of the institution’s budget 

(project funding is excluded) 

• Percent of the library’s budget from project funding (special grants) or income 
generation 

"Special grants" comprise all funding the library gets by special claims, not by the "normal” 
budget. 

This indicator assesses whether the library is successful in claiming additional funding or in 
generating income. 

• Number of training lessons per staff member 

The indicator assesses the average number of times each staff member has participated in 
training lessons. 

• Number of short-time illnesses per staff member 

A short-time illness is an absence from 1 to 3 days. 

Strategy with the Balanced Scorecard 

One great advantage of the Balanced Scorecard is that it can visualize relationships of cause and effect 
between target values, evaluation data and actions taken. 

The following example shows the planning process from the definition of goals and target values and the 
choice of adequate indicators to the actions that the library takes to achieve the target value. 



Figure 2: Managing with the Balanced Scorecard 

As the mission of academic libraries is in many aspects identical, the indicator system of the project might 
be used as reference model for benchmarking purposes. Individual varieties in libraries can be expressed 
by different target values and operational actions. Thus, a library whose main task is to provide basic 
information for students will further the use of electronic media by offering multimedia learning material. 
A special research library, on the other side, would perhaps offer its scientific journals in electronic form 
to achieve the same result. In spite of such differences, benchmarking would be possible. 

The implementation and continuous use of the Balanced Scorecard demands a large set of data. The 
project has developed a special tool named Library Audit based on a system of data analysis (OLAP) that 
allows the multidimensional and flexible analysis of data collections. 

The library in Munster has already filled Library Audit with several thousand data, classified as to the 
library’s products and services. Benchmarking data of other libraries are added continuously. Many of 
these data will not be used in the strategic evaluation of the Balanced Scorecard, but the large data pool 
can be useful for many operational problems. 
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The number of indicators for the Balanced Scorecard has been purposely kept small in order to avoid a 
flood of data without direct relevance for the strategic management. When choosing the indicators for the 
Balanced Scorecard the project libraries oriented themselves by the concept of the hybrid library that 
combines electronic and traditional library services in a comprehensive function. 

Structuring and implementing a scorecard model for a library demands a clear formulation of mission and 
strategic goals - a duty that has not yet been performed by every academic library. 

The most important issue in the integrated controlling concept is not to look at different quality aspects 
separately, but to keep them all in view. An example shows the steps of measuring quality in collection 
building: 

1 . The costs per document processed are low 

Does that mean that there are back-logs? 

2. Processing time proves quick and adequate. Processes are well organized. 

But perhaps there is no time for claiming overview orders? 

3. Claiming is done regularly and in good time. 

Maybe staff is overworked, and absence rates because of illness are rising? 

4. Illness rates are quite normal, and a staff satisfaction survey shows high satisfaction with the 
job: Everything looks fine. 

5. But: Collection use is going down, and a user survey shows dissatisfaction with the 
collection: 

Apparently, much well-organized labour has been spent on the wrong material. 

Service quality has many aspects - the Balanced Scorecard integrates them. 
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Figure 1 : The Balanced Scorecard 
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Figure 2: Managing with the Balanced Scorecard 
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